Interaction method and apparatus, display device, and storage medium

ABSTRACT

This application discloses an interaction method performed by a display device and the display device. A three-dimensional virtual object may be obtained; an optical signal of the virtual object is transmitted to a first region of a space and then projected from the first region into a second region of the space through the display device, so that an image of the virtual object is generated in the second region of the space that is outside the display device; when it is detected that a user performs a motion sensing operation on the image of the virtual object in the second region of the space, the virtual object is updated based on the motion sensing operation, to obtain an updated virtual object; and an optical signal of the updated virtual object is then used for generating an image of the updated virtual object in the second region of the space.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of PCT Patent ApplicationNo. PCT/CN2021/097660, entitled “INTERACTION METHOD AND APPARATUS,DISPLAY DEVICE, AND STORAGE MEDIUM” filed on Jun. 1, 2021, which claimspriority to Chinese Patent Application No. 202010683899.7, filed withthe State Intellectual Property Office of the People's Republic of Chinaon Jul. 16, 2020, and entitled “INTERACTION METHOD AND APPARATUS,DISPLAY DEVICE, AND STORAGE MEDIUM”, all of which are incorporatedherein by reference in their entirety.

FIELD OF THE TECHNOLOGY

This application relates to the field of computers, and in particular,to an interaction method and apparatus, a display device, and a storagemedium.

BACKGROUND OF THE DISCLOSURE

In recent years, with the development of three-dimensional imagingtechnologies, a three-dimensional imaging display and athree-dimensional imaging projector provide a new information outputmanner. Compared with conventional two-dimensional imaging,three-dimensional imaging has the advantages of rich details andconforming to the natural habits of human beings.

A current three-dimensional imaging technology is to project athree-dimensional virtual object on a transparent three-dimensionaldisplay screen for imaging, to achieve an effect in which thethree-dimensional virtual object is suspended in the air.

However, at present, a method of interacting with a three-dimensionalvirtual object by using a method of operating a projection device ortouching a three-dimensional display screen cannot meet requirements ofa user. Therefore, a current interaction method of a three-dimensionalvirtual object is single.

SUMMARY

Embodiments of this application provide an interaction method andapparatus, a display device, and a storage medium, so that a user mayperform motion sensing interaction with a 3D image projected in air.

An embodiment of this application provides an interaction method,performed by a display device. the method including:

obtaining a three-dimensional virtual object;

transmitting an optical signal of the three-dimensional virtual objectto a first region of a three-dimensional space and projecting theoptical signal of the three-dimensional virtual object from the firstregion into a second region of the space through the display device, sothat a three-dimensional image of the three-dimensional virtual objectis generated in the second region of the space that is outside thedisplay device;

when it is detected that a user performs a motion sensing operation onthe image of the three-dimensional virtual object in the second regionof the space, updating the three-dimensional virtual object based on themotion sensing operation, to obtain an updated three-dimensional virtualobject; and

transmitting an optical signal of the updated three-dimensional virtualobject to the first region of the space and projecting the opticalsignal of the updated three-dimensional virtual object from the firstregion into the second region of the space through the display device,so that an image of the updated three-dimensional virtual object isgenerated in the second region of the space.

An embodiment of this application further provides an interactionapparatus, including:

an obtaining unit, configured to obtain a three-dimensional virtualobject;

a transmitting unit, configured to transmit an optical signal of thethree-dimensional virtual object to a first region of athree-dimensional space and projecting the optical signal of thethree-dimensional virtual object from the first region into a secondregion of the space through the display device, so that athree-dimensional image of the three-dimensional virtual object isgenerated in the second region of the space that is outside the displaydevice;

A motion sensing unit, configured to update, when it is detected that auser performs a motion sensing operation on the image of thethree-dimensional virtual object in the second region of the space, thethree-dimensional virtual object based on the motion sensing operation,to obtain an updated three-dimensional virtual object; and

an interaction unit, configured to an optical signal of the updatedthree-dimensional virtual object to the first region of the space andprojecting the optical signal of the updated three-dimensional virtualobject from the first region into the second region of the space throughthe display device, so that an image of the updated three-dimensionalvirtual object is generated in the second region of the space.

An embodiment of this application further provides a display device,including a display module, an optical panel, a motion sensing detectionmodule, a memory, and a processor,

the motion sensing detection module being configured to detect a motionsensing operation of a user;

the display module being configured to transmit an optical signal;

the optical panel being disposed at a junction between a first regionand a second region of a space and being configured to negativelyrefract an optical signal of a three-dimensional virtual objecttransmitted to the first region of the space, so that an image of thethree-dimensional virtual object is generated in the second region ofthe space by using the optical signal;

the memory being configured to store data of the three-dimensionalvirtual object and a plurality of instructions; and

the processor being configured to read the plurality of instructionsstored in the memory to perform the following operations:

obtaining the three-dimensional virtual object;

controlling the display module to transmit the optical signal of thethree-dimensional virtual object to a first region of athree-dimensional space and projecting the optical signal of thethree-dimensional virtual object from the first region into a secondregion of the space through the display device, so that athree-dimensional image of the three-dimensional virtual object isgenerated in the second region of the space that is outside the displaydevice;

when it is detected that a user performs a motion sensing operation onthe image of the three-dimensional virtual object in the second regionof the space, updating the three-dimensional virtual object based on themotion sensing operation, to obtain an updated three-dimensional virtualobject; and

controlling the display module to transmit an optical signal of theupdated three-dimensional virtual object to the first region of thespace and projecting the optical signal of the updated three-dimensionalvirtual object from the first region into the second region of the spacethrough the display device, so that an image of the updatedthree-dimensional virtual object is generated in the second region ofthe space.

An embodiment of this application further provides a non-transitorycomputer-readable storage medium, storing a plurality of instructions,the instructions, when executed by a processor of a display device,causing the display device to perform the operations of any interactionmethod according to the embodiments of this application.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of thisapplication more clearly, the following briefly describes accompanyingdrawings required for describing the embodiments. Apparently, theaccompanying drawings in the following description show merely someembodiments of this application, and a person skilled in the art maystill derive other drawings from these accompanying drawings withoutcreative efforts.

FIG. 1A is a schematic diagram of a scenario of an interaction methodaccording to an embodiment of this application.

FIG. 1B is a schematic flowchart of an interaction method according toan embodiment of this application.

FIG. 1C is a schematic diagram of light negative refraction of aninteraction method according to an embodiment of this application.

FIG. 1D is a schematic diagram of imaging of an interaction methodaccording to an embodiment of this application.

FIG. 2A is a schematic diagram of a mechanical structure of a displaydevice according to an embodiment of this application.

FIG. 2B is a schematic flowchart of an application scenario of aninteraction method according to an embodiment of this application.

FIG. 3 is a schematic structural diagram of an interaction apparatusaccording to an embodiment of this application.

FIG. 4 is a schematic structural diagram of a display device accordingto an embodiment of this application.

DESCRIPTION OF EMBODIMENTS

The technical solutions in embodiments of this application are clearlyand completely described in the following with reference to theaccompanying drawings in the embodiments of this application.Apparently, the described embodiments are merely some rather than all ofthe embodiments of this application. All other embodiments obtained by aperson skilled in the art based on the embodiments of this applicationwithout creative efforts shall fall within the protection scope of thisapplication.

The embodiments of this application provide an interaction method andapparatus, a display device, and a storage medium.

The interaction apparatus may be specifically integrated in anelectronic device. The electronic device may be a device such as aprojector, a display, or a terminal.

For example, referring to FIG. 1A, the electronic device may be adisplay device. The display device may include a display module, anoptical panel, a motion sensing detection module, a memory, and aprocessor.

The motion sensing detection module is configured to detect a motionsensing operation of a user.

The display module is configured to transmit an optical signal.

The optical panel is disposed at a junction between a first region and asecond region of a space and is configured to negatively refract anoptical signal of a three-dimensional virtual object transmitted to thefirst region of the space, so that an image of the three-dimensionalvirtual object is generated in the second region of the space by usingthe optical signal.

The memory is configured to store data of the three-dimensional virtualobject and a plurality of instructions.

The processor is configured to read the plurality of instructions storedin the memory to perform the following operations:

obtaining the three-dimensional virtual object;

controlling the display module to transmit the optical signal of thethree-dimensional virtual object to the first region of the space, sothat the imaging of the three-dimensional virtual object is generated inthe second region of the space;

updating, when the motion sensing detection module detects that a userperforms a motion sensing operation on the image of thethree-dimensional virtual object in the second region of the space, thethree-dimensional virtual object based on the motion sensing operation,to obtain an updated three-dimensional virtual object; and

controlling the display module to transmit an optical signal of theupdated three-dimensional virtual object to the first region of thespace, so that an image of the updated three-dimensional virtual objectis generated in the second region of the space, to implement interactionbetween the user and the three-dimensional virtual object.

Detailed descriptions are separately provided below. Sequence numbers ofthe following embodiments are not intended to limit preference orders ofthe embodiments.

In this embodiment, a method for interacting with a three-dimensionalvirtual object is provided. A three-dimensional virtual object may bedisplayed in the air by using the interaction method, and a user mayinteract with the three-dimensional virtual object in the air, toachieve a tactile feedback effect in which an object is truly presentedin front of the user and the user can feel and touch the object.

As shown in FIG. 1B, a specific process of the interaction method may beas follows.

101. Obtain a three-dimensional virtual object.

Because there is a specific eye distance between a left eye and a righteye of a person, in a real three-dimensional world, images seen by theleft eye and the right eye of the person are not completely the same,and after imaging of two different images is generated in the eyes, astereoscopic sense of space is generated in a human brain.

The three-dimensional virtual object may include a left image and aright image. When a left eye of a person receives the left image and aright eye receives the right image, a sense of space of thethree-dimensional virtual object may be generate in a human brain.

The three-dimensional virtual object may be an image of athree-dimensional virtual character, an image of a three-dimensionalvirtual object, or the like.

There are a plurality of manners of obtaining the three-dimensionalvirtual object. For example, the three-dimensional virtual object isobtained from a server by using a network. In another example, thethree-dimensional virtual object is determined by reading athree-dimensional model stored locally. In another example, athree-dimensional virtual object of an object in a real world isacquired by using a holographic technology.

102. Transmit an optical signal of the three-dimensional virtual objectto a first region of a space, so that an image of the three-dimensionalvirtual object is generated in a second region of the space.

It is assumed that the technical solution of this application isapplicable to the game field. A display device shown in FIG. 2A is usedas an example, and FIG. 2A is a schematic diagram of a mechanicalstructure of a display device. The display device includes a gesturedetection module, a display module, a negative refraction panel, atactile feedback module, and a housing of the display device. Thehousing may also be referred to as a structural member and is made of ahard or flexible material. This is not limited herein.

The first region of the space is located inside the display device, thatis, formed by the negative refraction panel, the display module, and thestructural member forming a closed space together with the negativerefraction panel and the display module. The interference from anexternal light source may be reduced by using the closed space, toimprove a display effect of an image. The second region of the space islocated outside the display device, that is, an open or semi-open regionoutside the negative refraction panel. The region is open to a userside, so that the user may interact with a stereoscopically displayedimage by using a body part such as a hand. The second region of thespace is a sensing region located outside the display device. An opticalpanel may be disposed at a junction between the first region and thesecond region. The display module may be disposed in the first region,and the user may perform motion sensing interaction with the displaydevice in the second region. In this embodiment, the optical panel isthe negative refraction panel. The negative refraction panel is locatedat a junction of the inside and the outside of the display device andmay divide the inside and the outside of the display device.

A relative position relationship between the display module and thenegative refraction panel is not limited in FIG. 2A. During specificimplementation, an inclination angle between the display module and thenegative refraction panel is between 30° and 60°. Different inclinationangles between the display module and the negative refraction panelcauses different imaging positions of the three-dimensional virtualobject generated in the second region of the space. The inclinationangle between the display module and the negative refraction panel maybe adjusted according to different use scenarios of the product.

When the display module transmits an optical signal of thethree-dimensional virtual object to the first region of the space, theoptical signal may penetrate through the optical panel, for example, thenegative refraction panel, and is emitted to the second region, therebyachieving imaging outside the display device.

The display module may include a light-field display screen. Forexample, the light-field display screen may be formed by a plurality ofprojectors. Optical signals in different directions are projected to thesecond region outside the display device by using an array formed by theplurality of projectors, to implement projection of thethree-dimensional virtual object.

After the display module transmits an optical signal of a game characterto the negative refraction panel, the optical signal is refracted by thenegative refraction panel, and with the negative refraction panel as amirror surface, imaging may be performed at another positioncorresponding to an original imaging position inside the display device.

Due to an optical characteristic of the negative refraction panel, therefracted image is mirror-reverted. Therefore, in some embodiments,mirror flipping needs to be performed on a game character in advance, anoptical signal of the mirror-flipped game character is refracted byusing the negative refraction panel, and finally a correctthree-dimensional virtual object of the game character is displayed in asensing region.

The optical panel may be a panel structure made of a transparent ortranslucent material. The optical panel may be a plane panel or may be acurved panel, or the like.

In some embodiments, the optical panel is a negative refraction panel.The negative refraction panel is a transparent panel made of a negativerefraction material. The negative refraction material has a negativerefraction and is opposite to a conventional refraction. When a lightwave is incident on an interface of a negative refraction material froma positive refraction material, an incident wave and a refracted waveare located at a same side in a normal direction of the interface.

After the optical signal is negatively refracted by using the negativerefraction panel, light may be deflected in a direction opposite to anincident direction, so that an image of the three-dimensional virtualobject displayed by the display module may be generated outside thedisplay device, and the user can see the three-dimensional virtualobject suspended in the air.

For example, an upper half part in FIG. 1C is a schematic diagram inwhich a light wave is incident on a glass panel with a refraction of 1.3from the air. An incident wave and a refracted wave are located onopposite sides in a normal direction of an interface. Therefore, afteran optical signal passes through a glass panel, imaging cannot beperformed in the air behind the other side of the glass panel. A lowerhalf part in FIG. 1C is a schematic diagram in which a light wave isincident on a negative refraction panel with a refraction of −1.3 fromthe air. An incident wave and a refracted wave are located at a sameside in a normal direction of an interface. Therefore, after an opticalsignal passes through the negative refraction panel, imaging can beperformed in the air behind the other side of the negative refractionpanel.

In some embodiments, the optical panel may be a panel module formed bypanels made of different materials. The panel module may include anegative refraction panel, a protecting glass, an optical auxiliarypanel, and the like.

Therefore, due to the characteristics of the negative refraction panel,a mirror image of the original three-dimensional virtual object isdisplayed after the optical signal passes through the negativerefraction panel. Therefore, to ensure the display accuracy of thethree-dimensional virtual object, mirror flipping needs to be performedon the original three-dimensional virtual object in advance in step 102,so that after an optical signal of a mirror-flipped three-dimensionalvirtual object passes through the negative refraction panel, theoriginal three-dimensional virtual object is displayed.

For example, in some embodiments, Step 102 may include the followingsteps.

(1) performing a mirror flipping operation on the three-dimensionalvirtual object, to obtain a flipped three-dimensional virtual object;and

(2) generating an optical signal of the flipped three-dimensionalvirtual object, and transmitting the optical signal of the flippedthree-dimensional virtual object to the first region of the space, anoptical panel being disposed in the first region of the space, so thatthe three-dimensional virtual object is displayed in the second regionof the space after the optical signal is negatively refracted by theoptical panel.

For example, referring to FIG. 1D, an upper part in FIG. 1D shows aneffect of directly transmitting an optical signal of an originalthree-dimensional virtual object, and a user may directly see theoriginal three-dimensional virtual object. A middle part in FIG. 1Dshows an effect of transmitting an optical signal of an originalthree-dimensional virtual object to a negative refraction panel, and auser may see a mirror image of the original three-dimensional virtualobject. A lower part in FIG. 1D shows an effect of first performingmirror flipping on an original three-dimensional virtual object and thentransmitting an optical signal of a flipped three-dimensional virtualobject to a negative refraction panel, and a user may see the originalthree-dimensional virtual object.

For example, referring to FIG. 1D, in some embodiments, because athree-dimensional virtual object is divided into a left image and aright image, an optical signal of the three-dimensional virtual objectmay be divided into a left optical signal and a right optical signal.Step (2) of generating an optical signal of the flippedthree-dimensional virtual object and transmitting the optical signal ofthe flipped three-dimensional virtual object to the first region of thespace may include the following steps:

generating the left optical signal and the right optical signal of theflipped three-dimensional virtual object; and

transmitting the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of a user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

103. Update, when it is detected that a user performs a motion sensingoperation on the image of the three-dimensional virtual object in thesecond region of the space, the three-dimensional virtual object basedon the motion sensing operation, to obtain an updated three-dimensionalvirtual object.

The motion sensing operation is an operation in which a person directlyinteracts with a display device by using a body action. Motion sensing,or referred to as somatosensory, may include a touch sense, a pressuresense, a sense of heat, a sense of pain, and the like. Different sensorsmay sense motion sensing of different types.

For example, the motion sensing operation may be used for capturing bodymovements of a human body by using a motion sensing sensor such as acamera, a handle, a foot strap, or an induction cap, to implement ahuman-machine interaction operation by using the body movements.

For example, when it is detected that a user “touches” a virtualthree-dimensional character in a second region of a space, the virtualthree-dimensional character may retreat.

In some embodiments, the motion sensing operation on the image of thethree-dimensional virtual object performed by the user in the secondregion of the space may be detected by using the motion sensingdetection module. Step 103 may include the following steps:

(1) determining an image coordinate set of the three-dimensional virtualobject in a virtual coordinate system;

(2) generating, when the motion sensing operation of the user in thesecond region of the space is detected, a motion sensing coordinate setof the user in the virtual coordinate system based on the motion sensingoperation;

(3) determining a touch status of the user for the imaging of thethree-dimensional virtual object based on the motion sensing coordinateset of the user and the image coordinate set; and

(4) updating the three-dimensional virtual object based on the touchstatus, to obtain the updated three-dimensional virtual object.

The virtual coordinate system is a preset three-dimensional coordinatesystem. The virtual coordinate system may be used for determining howthe motion sensing operation of the user in reality acts on thethree-dimensional virtual object in the virtual scene.

For example, it is assumed that an image coordinate set of athree-dimensional virtual object in the virtual coordinate system is P,P=p(x_(i), y_(i), p(x_(i), y_(i), z_(i)) being image coordinates of eachpixel of the three-dimensional virtual object in the virtual coordinatesystem, i being a positive integer greater than 0.

When a motion sensing operation of the user in the second region of thespace is detected, a motion sensing coordinate set of the user may begenerated in the virtual coordinate system based on the motion sensingoperation. For example, when a gesture operation of the user in thesecond region of the space is detected, a motion sensing coordinate setH of a hand of the user in the virtual coordinate system may bedetermined, H=h(x_(j), y_(j), z_(j)), h(x_(j), y_(j), z_(j)) being imagecoordinates of each skeleton node of the hand of the user in the virtualcoordinate system, j being a positive integer greater than 0.

When P intersects H, it may be determined that the hand of the usertouches the three-dimensional virtual object. Therefore, it may bedetermined that a touch status of the user for imaging of thethree-dimensional virtual object is [touch]. When P does not intersectH, it may be determined that the hand of the user does not touch thethree-dimensional virtual object. Therefore, it may be determined thatthe touch status of the user for the imaging of the three-dimensionalvirtual object is [not touch].

When it is determined that the hand of the user touches thethree-dimensional virtual object, step 103 of updating and displayingmay be performed on the three-dimensional virtual object.

Specifically, in some embodiments, the motion sensing coordinate set mayinclude a skeleton node coordinate set, and step (2) of generating, whenthe motion sensing operation of the user in the second region of thespace is detected, a motion sensing coordinate set of the user in thevirtual coordinate system based on the motion sensing operation mayinclude the following steps:

obtaining a hand image of the user in the second region of the space;

performing skeletal analysis on the hand image, to determine skeletonnodes of a hand of the user; and

determining coordinates of the skeleton nodes of the hand of the user inthe virtual coordinate system, to obtain the skeleton node coordinateset.

For example, the motion sensing detection module may acquire images inthe second region, and determine whether the images include a hand imageof the user by performing image classification on the images.

Subsequently, the motion sensing detection module may recognize skeletonnodes of a hand of the user in the hand image, to determine coordinatesof the skeleton nodes of the hand of the user in the virtual coordinatesystem.

The motion sensing detection module may acquire a depth image, a leftimage, a right image, and the like.

For example, the motion sensing detection module may include a leftcamera and a right camera. The motion sensing detection module mayconstruct a depth image based on a left image and a right image acquiredby the left camera and the right camera in the second region, andperform image semantic segmentation on the depth image by using a fullyconnected neural network, to obtain a hand depth image of the hand ofthe user from the depth image. Subsequently, each skeleton node of thehand of the user in the hand depth image is determined by performingimage recognition on the hand depth image, to determine coordinates ofthe skeleton nodes of the hand of the user in the virtual coordinatesystem, thereby obtaining a motion sensing coordinate set of the user.

A gesture of the user, whether the hand of the user intersects with thethree-dimensional virtual object in the virtual coordinate system, andthe like may be determined by using the motion sensing coordinate set ofthe user, that is, coordinates of each skeleton node of the hand of theuser in the virtual coordinate system.

In some embodiments, after the hand of the user enters the secondregion, the display device may correspondingly adjust an action of thevirtual three-dimensional character according to a change of the hand ofthe user, to implement visual interaction.

In addition, in some embodiments, to give tactile perception to the userand further improve user experience, in some embodiments, step 103 mayfurther include the following steps:

(1) determining motion sensing feedback information corresponding to themotion sensing operation; and

(2) transmitting the motion sensing feedback information to the secondregion of the space, so that the user receives the motion sensingfeedback information in the second region of the space.

Motion sensing feedback refers to giving somatosensory such a touchsense, a pressure sense, a sense of heat, a sense of heat, or the liketo the user. For example, the motion sensing feedback may be classifiedas tactile feedback, force feedback, optical feedback, and the likeaccording to types.

In this solution, motion sensing feedback information may be transmittedto the second region of the space by using the motion sensing feedbackmodule.

The motion sensing feedback module may include a tactile feedbackgenerator, a force feedback generator, an optical feedback generator,and the like.

For example, the tactile feedback generator may include an ultrasonictactile feedback generator, an infrared tactile feedback generator, andthe like.

The ultrasonic tactile feedback generator may emit an ultrasonic energybeam, to cause the ultrasonic energy beam to strike a skin of a person,so that the person feels tactile feedback.

For example, in some embodiments, to allow the user to feel tactilefeedback in addition to visual changes, so as to achievemulti-dimensional interactive feedback experience, the motion sensingfeedback may include tactile feedback, the motion sensing feedbackinformation may include an ultrasonic signal, and step (1) ofdetermining motion sensing feedback information corresponding to themotion sensing operation may include the following steps:

generating the ultrasonic signal based on the motion sensing operation.

Therefore, step (2) of transmitting the motion sensing feedbackinformation to the second region of the space, so that the user receivesthe motion sensing feedback information in the second region of thespace may include the following steps:

transmitting the ultrasonic signal to the second region of the space, sothat the user feels the tactile feedback formed by the ultrasonic signalin the second region of the space.

Step 104. Transmit an optical signal of the updated three-dimensionalvirtual object to the first region of the space, so that an image of theupdated three-dimensional virtual object is generated in the secondregion of the space, to implement interaction between the user and thethree-dimensional virtual object.

In some embodiments, step 104 may include the following steps:

(1) performing a mirror flipping operation on the updatedthree-dimensional virtual object, to obtain a flipped updatedthree-dimensional virtual object; and

(2) generating an optical signal of the flipped updatedthree-dimensional virtual object, and transmitting the optical signal ofthe flipped updated three-dimensional virtual object to the first regionof the space, an optical panel being disposed in the first region of thespace, so that the updated three-dimensional virtual object is displayedin the second region of the space after the optical signal of theflipped updated three-dimensional virtual object is negatively refractedby the optical panel.

In some embodiments, the optical signal may include a left opticalsignal and a right optical signal, and step (2) of generating an opticalsignal of the flipped updated three-dimensional virtual object, andtransmitting the optical signal of the flipped updated three-dimensionalvirtual object to the first region of the space may include thefollowing steps:

generating the left optical signal and the right optical signal of theflipped updated three-dimensional virtual object; and

transmitting the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of a user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

For the step, reference may be made to step 102, and details are notdescribed herein again.

It can be learned from the above that in this embodiment of thisapplication, a three-dimensional virtual object may be obtained; anoptical signal of the three-dimensional virtual object is transmitted toa first region of a space, so that an image of the three-dimensionalvirtual object is generated in a second region of the space; when it isdetected that a user performs a motion sensing operation on the image ofthe three-dimensional virtual object in the second region of the space,the three-dimensional virtual object is updated based on the motionsensing operation, to obtain an updated three-dimensional virtualobject; and an optical signal of the updated three-dimensional virtualobject is transmitted to the first region of the space, so that an imageof the updated three-dimensional virtual object is generated in thesecond region of the space, to implement interaction between the userand the three-dimensional virtual object.

Therefore, this solution provides a manner of displaying athree-dimensional virtual object in a sensing region in the air. Becausea physical limitation of only relying on a screen surface as a displayboundary is eliminated, a user can perform motion sensing interactionwith the three-dimensional virtual object in a sensing region, where thesensing region may be configured with various combinations of sensingsensors and feedback generators, including but not limited to, aninfrared sensor, a time of flight (TOF) sensor, a binocular camerasensor, an ultrasonic sensor, and the like, and also including, but notlimited to, an ultrasonic feedback generator, an electromagneticfeedback generator, and the like, thereby improving user experience andimplementing a more convenient and more dimensional manner to interactwith a displayed image. Therefore, the interaction method provided inthis solution enables the user to perform motion sensing interactionwith a 3D image projected in the air.

According to the method described in the foregoing embodiments, thefollowing further provides a detailed description.

In this embodiment, the method in this embodiment of this application isdescribed in detail by using an example in which this application isapplicable to the game field and is applicable to a display device shownin FIG. 2A.

As shown in FIG. 2B, a specific process of an interaction method is asfollows.

201. A display module transmits an optical signal of a game character toa negative refraction panel, so that a three-dimensional virtual objectof the game character is generated outside a display device.

For a specific display method, reference may be made to step 102.Details are not described herein.

202. When a gesture detection module detects that a hand of a playertouches imaging of the game character outside the display device, changean action of the game character based on the touch of the hand of theplayer, to obtain a changed game character.

The gesture detection module may adopt, but not limited to, an existinginfrared gesture detection solution, an ultrasonic gesture detectionsolution, or the like.

The gesture detection module determines a corresponding interactionoperation by detecting a movement track and a gesture state of a hand ofa user in a sensing region, so that an action of a game character iscorrespondingly changed according to the interaction operation.

In some embodiments, a negative refraction panel may be obliquely placedrelative to a display module, and an inclination angle of the negativerefraction panel may be changed according to an actual requirement, tomodify an imaging effect, an imaging position, and the like of imagingof the game character.

In some embodiments, to facilitate a user to adjust an imaging positionand an imaging effect of imaging, a mechanical structure of the displaydevice may further include inclination angle adjusting grooves of aplurality of inclination angles of the negative refraction panel. Thenegative refraction panel is fixed to different inclination angleadjusting grooves, so that the negative refraction panel may beobliquely placed at different inclination angles relative to the displaymodule.

For a specific detection method, reference may be made to step 103.Details are not described herein.

203. The display module transmits an optical signal of the changed gamecharacter to the negative refraction panel, so that the action of thegame character in the three-dimensional virtual object displayed outsidethe device is changed.

For example, when a user tries to touch a three-dimensional virtualobject of a game character, the game character may generate acorresponding action change, for example, moving, dodging, deforming,disappearing, and other actions.

For a specific display method, reference may be made to step 101.Details are not described herein.

204. A motion sensing feedback module transmits an ultrasonic signal tooutside the display device, so that the player feels a touch sensegenerated by the ultrasonic signal touching a skin, to implementinteraction between a user and the game character.

For example, the motion sensing feedback module may include, but notlimited to, an ultrasonic energy generator, an infrared energygenerator, and the like. For example, an ultrasonic energy beam may beemitted by using the ultrasonic energy generator, so that the ultrasonicenergy beam strikes a skin surface of a hand of a user, to cause theuser to feel tactile feedback.

For a specific tactile feedback method, reference may be made to step103. Details are not described herein.

Therefore, in this solution, a display device may display athree-dimensional virtual object of a game character in a sensing regionoutside the display device, to eliminate a physical limitation of onlyrelying on a screen surface as a display boundary. After a hand of auser enters the sensing region outside the display device, the displaydevice correspondingly adjusts an action of the game character accordingto a change of the hand of the user, to implement visual interaction. Inaddition, a tactile feedback generator may also give correspondingtactile perception to the hand of the user, thereby implementing visualand tactile interactive perception.

It can be learned from the above that in this solution, a display modulemay transmit an optical signal of a game character to a negativerefraction panel, so that a three-dimensional virtual object of the gamecharacter is generated outside a display device; when a gesturedetection module detects that a hand of a player touches imaging of thegame character outside the display device, an action of the gamecharacter is changed based on the touch of the hand of the user, toobtain a changed game character; the display module transmits an opticalsignal of the changed game character to the negative refraction panel,so that the action of the game character in the three-dimensionalvirtual object displayed outside the display device is changed; and amotion sensing feedback module transmit an ultrasonic signal to outsidethe display device, so that the player feels a touch sense generated bythe ultrasonic signal touching a skin, to implement interaction betweena user and the game character.

In this application, a user may interact in the second region with aprojection of a 3D image originally projected in the first region.Therefore, the interaction method provided in this solution enables theuser to perform motion sensing interaction with a 3D image projected inthe air.

To better implement the method, an embodiment of this applicationfurther provides an interaction apparatus. The interaction apparatus maybe specifically integrated in an electronic device. The electronicdevice may be a device such as a display, a projector, or a terminal.

For example, in this embodiment, by using an example in which theinteraction apparatus is integrated in the display, the interactionapparatus in this embodiment of this application is described in detail.

For example, as shown in FIG. 3, the interaction apparatus may includean obtaining unit 301, a transmitting unit 302, a motion sensing unit303, and an interaction unit 304 as follows.

(1) Obtaining unit 301:

the obtaining unit 301 may be configured to obtain a three-dimensionalvirtual object.

(2) Transmitting unit 302:

the transmitting unit 302 may be configured to transmit an opticalsignal of the three-dimensional virtual object to a first region of aspace, so that an image of the three-dimensional virtual object isgenerated in a second region of the space.

In some embodiments, the transmitting unit 302 may include a flippingsubunit and a transmitting subunit as follows.

(1) Flipping Subunit:

the flipping subunit may be configured to perform a mirror flippingoperation on the three-dimensional virtual object, to obtain a flippedthree-dimensional virtual object.

(2) Transmitting Subunit:

the transmitting subunit may be configured to generate an optical signalof the flipped three-dimensional virtual object, and transmit theoptical signal of the flipped three-dimensional virtual object to thefirst region of the space, an optical panel being disposed in the firstregion of the space, so that the three-dimensional virtual object isdisplayed in the second region of the space after the optical signal isnegatively refracted by the optical panel.

In some embodiments, the optical signal may include a left opticalsignal and a right optical signal, and the transmitting subunit may beconfigured to:

generate the left optical signal and the right optical signal of theflipped three-dimensional virtual object; and

transmit the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of a user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

(3) Motion Sensing Unit 303:

the motion sensing unit 303 may be configured to update, when it isdetected that a user performs a motion sensing operation on the image ofthe three-dimensional virtual object in the second region of the space,the three-dimensional virtual object based on the motion sensingoperation, to obtain an updated three-dimensional virtual object.

In some embodiments, the motion sensing unit 303 may include an imagecoordinate subunit, a motion sensing coordinate subunit, a touch statussubunit, and an updating subunit as follows.

(1) Image Coordinate Subunit:

the image coordinate subunit may be configured to determine an imagecoordinate set of the three-dimensional virtual object in a virtualcoordinate system.

(2) Motion Sensing Subunit:

the motion sensing subunit may be configured to generate, when themotion sensing operation of the user in the second region of the spaceis detected, a motion sensing coordinate set of the user in the virtualcoordinate system based on the motion sensing operation.

In some embodiments, the motion sensing coordinate set may include askeleton node coordinate set, and the motion sensing coordinate subunitmay be configured to:

obtain a hand image of the user in the second region of the space;

perform skeletal analysis on the hand image, to determine skeleton nodesof a hand of the user; and

determine coordinates of the skeleton nodes of the hand of the user inthe virtual coordinate system, to obtain the skeleton node coordinateset.

(3) Touch Status Subunit:

the touch status subunit may be configured to determine a touch statusof the user for the imaging of the three-dimensional virtual objectbased on the motion sensing coordinate set of the user and the imagecoordinate set.

(4) Updating Subunit:

the updating subunit may be configured to update the three-dimensionalvirtual object based on the touch status, to obtain the updatedthree-dimensional virtual object.

In some embodiments, the motion sensing unit 303 may further include afeedback information subunit and a feedback subunit as follows.

(1) Feedback Information Subunit:

the feedback information subunit may be configured to determine motionsensing feedback information corresponding to the motion sensingoperation.

In some embodiments, the motion sensing feedback may include tactilefeedback, the feedback information may include an ultrasonic signal, andthe feedback information subunit may be configured to:

generate the ultrasonic signal based on the motion sensing operation.

(2) Feedback Subunit:

the feedback subunit may be configured to transmit the motion sensingfeedback information to the second region of the space, so that the userreceives the motion sensing feedback information in the second region ofthe space.

In some embodiments, the feedback subunit may be configured to:

transmit the ultrasonic signal to the second region of the space, sothat the user feels the tactile feedback formed by the ultrasonic signalin the second region of the space.

(4) Interaction Unit 304:

the interaction unit 304 may be configured to transmit an optical signalof the updated three-dimensional virtual object to the first region ofthe space, so that an image of the updated three-dimensional virtualobject is generated in the second region of the space, to implementinteraction between the user and the three-dimensional virtual object.

In some embodiments, the interaction unit 304 may include an updateflipping subunit and an update transmitting subunit as follows.

(1) Update Flipping Subunit:

the update flipping subunit may be configured to perform a mirrorflipping operation on the updated three-dimensional virtual object, toobtain a flipped updated three-dimensional virtual object.

(2) Update Transmitting Subunit:

the update transmitting subunit may be configured to generate an opticalsignal of the flipped updated three-dimensional virtual object, andtransmit the optical signal of the flipped updated three-dimensionalvirtual object to the first region of the space, an optical panel beingdisposed in the first region of the space, so that the updatedthree-dimensional virtual object is displayed in the second region ofthe space after the optical signal of the flipped updatedthree-dimensional virtual object is negatively refracted by the opticalpanel.

In some embodiments, the optical signal may include a left opticalsignal and a right optical signal, and the update transmitting subunitmay be configured to:

generate the left optical signal and the right optical signal of theflipped updated three-dimensional virtual object; and

transmit the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of a user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

During specific implementations, the foregoing units may be implementedas independent entities, or may be randomly combined, or may beimplemented as the same entity or several entities. For specificimplementations of the foregoing units, refer to the foregoing methodembodiments. Details are not described herein again.

It can be learned from the above that according to the interactionapparatus in this embodiment, an obtaining unit obtains athree-dimensional virtual object; a transmitting unit transmits anoptical signal of the three-dimensional virtual object to a first regionof a space, so that an image of the three-dimensional virtual object isgenerated in a second region of the space; a motion sensing unitupdates, when it is detected that a user performs a motion sensingoperation on the image of the three-dimensional virtual object in thesecond region of the space, the three-dimensional virtual object basedon the motion sensing operation, to obtain an updated three-dimensionalvirtual object; and an interaction unit transmits an optical signal ofthe updated three-dimensional virtual object to the first region of thespace, so that an image of the updated three-dimensional virtual objectis generated in the second region of the space, to implement interactionbetween the user and the three-dimensional virtual object.

Therefore, in this embodiment of this application, a user may performmotion sensing interaction with a 3D image projected in the air.

An embodiment of this application further provides an electronic device.The electronic device may be a display device such as a display, aprojector, or a display terminal.

In this embodiment, a detailed description is made by using an examplein which the electronic device of this embodiment is the display device.For example, FIG. 4 is a schematic structural diagram of a displaydevice according to an embodiment of this application. Specifically:

The display device may include components such as a processor 401 withone or more processing cores, a memory 402 with one or morecomputer-readable storage media, a power supply 403, a display module404, a motion sensing detection module 405, and a motion sensingfeedback module 406. In addition, the display device may further includean optical panel. The optical panel may not be electrically connected toeach part of the entire display device. The optical panel may be a panelcomponent including a negative refraction panel. The negative refractionpanel is a panel made of a negative refraction material.

A dielectric constant or a permeability of the negative refractionmaterial is negative and has a negative refraction.

A person skilled in the art may understand that the structure of thedisplay device shown in FIG. 4 does not constitute a limitation to thedisplay device, and the display device may include more components orfewer components than those shown in the figure, or some components maybe combined, or a different component deployment may be used.

The processor 401 is a control center of the display device, and isconnected to various parts of the entire display device by using variousinterfaces and lines. By running or executing the software programand/or module stored in the memory 402, and invoking data stored in thememory 402, the processor performs various functions and data processingof the device, thereby performing overall monitoring on the displaydevice. In some embodiments, the processor 401 may include one or moreprocessing cores. In some embodiments, the processor 401 may integratean application processor and a modem. The application processor mainlyprocesses an operating system, a user interface, an application program,and the like. The modem mainly processes wireless communication. It maybe understood that the foregoing modem may either not be integrated intothe processor 401.

The memory 402 may be configured to store a software program and amodule, and the processor 401 runs the software program and the modulethat are stored in the memory 402, to implement various functionalapplications and data processing. The memory 402 may mainly include aprogram storage area and a data storage area. The program storage areamay store an operating system, an application program required by atleast one function (for example, a sound playback function and an imageplayback function), or the like. The data storage area may store datacreated according to use of the display device. In addition, the memory402 may include a high speed random access memory, and may also includea non-volatile memory, such as at least one magnetic disk storagedevice, a flash memory, or another volatile solid-state storage device.Correspondingly, the memory 402 may further include a memory controller,to provide access of the processor 401 to the memory 402.

The display device further includes the power supply 403 supplying powerto various components. In some embodiments, the power supply 403 may belogically connected to the processor 401 by using a power managementsystem, thereby implementing functions such as charging, discharging,and power consumption management by using the power management system.The power supply 403 may further include one or more of a direct currentor alternating current power supply, a re-charging system, a powerfailure detection circuit, a power supply converter or inverter, a powersupply state indicator, and any other component.

The display device may further include the display module 404. Thedisplay module 404 may be configured to transmit an optical signal andthe like. In some embodiments, the display module 404 may include alight-field display screen, a polarized display screen, and the like.

The display device may further include the motion sensing detectionmodule 405. The motion sensing detection module 405 may be configured todetect a motion sensing operation of a user. The motion sensingdetection module 405 may include an infrared sensor, an ultrasonicsensor, an image sensor, a depth sensor, and the like.

The display device may further include the motion sensing feedbackmodule 406. The motion sensing feedback module 406 is configured totransmit motion sensing feedback information. The motion sensingfeedback information may include ultrasonic information, infraredinformation, and the like.

Although not shown in the figure, the display device may further includea sound unit, and the like. Details are not described herein again.

Specifically, in this embodiment, the processor 401 in the displaydevice may load executable files corresponding to processes of one ormore application programs to the memory 402 according to the followinginstructions, and the processor 401 runs the application program storedin the memory 402, to implement various functions as follows:

obtaining a three-dimensional virtual object;

transmitting an optical signal of the three-dimensional virtual objectto a first region of a space, so that an image of the three-dimensionalvirtual object is generated in a second region of the space;

updating, when it is detected that a user performs a motion sensingoperation on the image of the three-dimensional virtual object in thesecond region of the space, the three-dimensional virtual object basedon the motion sensing operation, to obtain an updated three-dimensionalvirtual object; and

transmitting an optical signal of the updated three-dimensional virtualobject to the first region of the space, so that an image of the updatedthree-dimensional virtual object is generated in the second region ofthe space, to implement interaction between the user and thethree-dimensional virtual object.

In some embodiments, the processor 401 is further configured to performthe following steps:

determining motion sensing feedback information corresponding to themotion sensing operation; and

controlling the motion sensing feedback module 406 to transmit themotion sensing feedback information to the second region of the space,so that the user receives the motion sensing feedback information in thesecond region of the space.

In some embodiments, the motion sensing feedback includes tactilefeedback, the motion sensing feedback information includes an ultrasonicsignal, and before the step of determining motion sensing feedbackinformation corresponding to the motion sensing operation is performed,the processor 401 is configured to perform the following steps:

generating the ultrasonic signal based on the motion sensing operation.

when the step of controlling the display module 404 to transmit themotion sensing feedback information to the second region of the space,so that the user receives the motion sensing feedback information in thesecond region of the space is performed, the processor 401 is configuredto perform the following steps:

controlling the display module 404 to transmit the ultrasonic signal tothe second region of the space, so that the user feels the tactilefeedback formed by the ultrasonic signal in the second region of thespace.

In some embodiments, when the step of controlling the display module 404to transmit an optical signal of the three-dimensional virtual object tothe first region of the space, so that an image of the three-dimensionalvirtual object is generated in the second region of the space isperformed, the processor 401 is configured to perform the followingsteps:

performing a mirror flipping operation on the three-dimensional virtualobject, to obtain a flipped three-dimensional virtual object; and

controlling the display module 404 to generate an optical signal of theflipped three-dimensional virtual object, and transmitting the opticalsignal of the flipped three-dimensional virtual object to the firstregion of the space, so that the three-dimensional virtual object isdisplayed in the second region of the space after the optical signal isnegatively refracted by the optical panel.

In some embodiments, the optical signal includes a left optical signaland a right optical signal, and when the step of controlling the displaymodule 404 to generate an optical signal of the flippedthree-dimensional virtual object, and transmitting the optical signal ofthe flipped three-dimensional virtual object to the first region of thespace is performed, the processor 401 is configured to perform thefollowing steps:

generating the left optical signal and the right optical signal of theflipped three-dimensional virtual object; and

transmitting the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of the user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

In some embodiments, when the step of transmitting the optical signal ofthe updated three-dimensional virtual object to the first region of thespace, so that an image of the updated three-dimensional virtual objectis generated in the second region of the space is performed, theprocessor 401 is configured to perform the following steps:

performing a mirror flipping operation on the updated three-dimensionalvirtual object, to obtain a flipped updated three-dimensional virtualobject; and

controlling the display module 404 to generate an optical signal of theflipped updated three-dimensional virtual object, and transmitting theoptical signal of the flipped updated three-dimensional virtual objectto the first region of the space, an optical panel being disposed in thefirst region of the space, so that the updated three-dimensional virtualobject is displayed in the second region of the space after the opticalsignal of the flipped updated three-dimensional virtual object isnegatively refracted by the optical panel.

In some embodiments, the optical signal includes a left optical signaland a right optical signal, and when the step of controlling the displaymodule 404 to generate an optical signal of the flipped updatedthree-dimensional virtual object and transmitting the optical signal ofthe flipped updated three-dimensional virtual object to the first regionof the space is performed, the processor 401 is configured to performthe following steps:

controlling the display module 404 to generate the left optical signaland the right optical signal of the flipped updated three-dimensionalvirtual object; and

transmitting the left optical signal and the right optical signal to thefirst region of the space, so that the left optical signal enters aright eye of a user after being negatively refracted by the opticalpanel and the right optical signal enters a left eye of the user afterbeing negatively refracted by the optical panel.

In some embodiments, when the step of updating, when it is detected thata user performs a motion sensing operation on the image of thethree-dimensional virtual object in the second region of the space, thethree-dimensional virtual object based on the motion sensing operation,to obtain an updated three-dimensional virtual object is performed, theprocessor 401 is configured to perform the following steps:

determining an image coordinate set of the three-dimensional virtualobject in a virtual coordinate system;

generating, when the motion sensing detection module 405 detects themotion sensing operation of the user in the second region of the space,a motion sensing coordinate set of the user in the virtual coordinatesystem based on the motion sensing operation;

determining a touch status of the user for the imaging of thethree-dimensional virtual object based on the motion sensing coordinateset of the user and the image coordinate set; and

updating the three-dimensional virtual object based on the touch status,to obtain the updated three-dimensional virtual object.

In some embodiments, the motion sensing coordinate set includes askeleton node coordinate set, and when the step of generating, when themotion sensing detection module 405 detects the motion sensing operationof the user in the second region of the space, a motion sensingcoordinate set of the user in the virtual coordinate system based on themotion sensing operation is performed, the motion sensing detectionmodule 405 is configured to perform the following steps:

obtaining a hand image of the user in the second region of the space;

performing skeletal analysis on the hand image, to determine skeletonnodes of a hand of the user; and

determining coordinates of the skeleton nodes of the hand of the user inthe virtual coordinate system, to obtain a skeleton node coordinate set.

For specific implementations of the above operations, reference may bemade to the foregoing embodiments. Details are not described hereinagain.

It can be learned from the above that according to the interactionmethod provided in this solution, a user may perform motion sensinginteraction with a 3D image in the air.

A person of ordinary skill in the art may understand that, all or somesteps of the methods in the foregoing embodiments may be implemented byusing instructions, or implemented through instructions controllingrelevant hardware, and the instructions may be stored in acomputer-readable storage medium and loaded and executed by a processor.

Accordingly, an embodiment of this application provides acomputer-readable storage medium, storing a plurality of instructions.The instructions can be loaded by a processor, to perform the steps inthe interaction method according to the embodiments of this application.For example, the instructions may perform the following steps:

obtaining a three-dimensional virtual object;

transmitting an optical signal of the three-dimensional virtual objectto a first region of a space, so that an image of the three-dimensionalvirtual object is generated in a second region of the space;

updating, when it is detected that a user performs a motion sensingoperation on the image of the three-dimensional virtual object in thesecond region of the space, the three-dimensional virtual object basedon the motion sensing operation, to obtain an updated three-dimensionalvirtual object; and

transmitting an optical signal of the updated three-dimensional virtualobject to the first region of the space, so that an image of the updatedthree-dimensional virtual object is generated in the second region ofthe space, to implement interaction between the user and thethree-dimensional virtual object.

According to an aspect of this application, a computer program productis provided, the computer program product or a computer programincluding computer instructions, the computer instructions being storedin a computer-readable storage medium. A processor of a computer devicereads the computer instructions from the computer-readable storagemedium, and executes the computer instructions, so that the computerdevice performs the method provided in the foregoing embodiments in athree-dimensional virtual object display aspect or a three-dimensionalvirtual object interaction aspect.

The storage medium may include a read-only memory (ROM), a random accessmemory (RAM), a magnetic disk, an optical disc, or the like.

Because the instructions stored in the storage medium may perform thesteps of any interaction method provided in the embodiments of thisapplication, the instructions can implement beneficial effects that maybe implemented by any interaction method provided in the embodiments ofthis application. For details, reference may be made to the foregoingembodiments. Details are not described herein again.

The interaction method and apparatus, the display device, and thecomputer-readable storage medium provided in the embodiments of thisapplication are described in detail above. The principles andimplementations of this application are described through specificexamples in this specification, and the descriptions of the embodimentsare only intended to help understand the methods and core ideas of thisapplication. In sum, the term “unit” or “module” in this applicationrefers to a computer program or part of the computer program that has apredefined function and works together with other related parts toachieve a predefined goal and may be all or partially implemented byusing software, hardware (e.g., processing circuitry and/or memoryconfigured to perform the predefined functions), or a combinationthereof. Each unit or module can be implemented using one or moreprocessors (or processors and memory). Likewise, a processor (orprocessors and memory) can be used to implement one or more modules orunits. Moreover, each module or unit can be part of an overall modulethat includes the functionalities of the module or unit. Meanwhile, aperson of ordinary skill in the art may make modifications to thespecific implementations and application scopes according to the ideasof the present disclosure. In conclusion, the content of thespecification is not to be construed as a limitation to the presentdisclosure.

What is claimed is:
 1. A method performed by a display device, themethod comprising: obtaining a three-dimensional virtual object; andtransmitting an optical signal of the three-dimensional virtual objectto a first region of a three-dimensional space and projecting theoptical signal of the three-dimensional virtual object from the firstregion into a second region of the space through the display device, sothat a three-dimensional image of the three-dimensional virtual objectis generated in the second region of the space that is outside thedisplay device.
 2. The method according to claim 1, the methodcomprising: when it is detected that a user performs a motion sensingoperation on the image of the three-dimensional virtual object in thesecond region of the space, updating the three-dimensional virtualobject based on the motion sensing operation, to obtain an updatedthree-dimensional virtual object; and transmitting an optical signal ofthe updated three-dimensional virtual object to the first region of thespace and projecting the optical signal of the updated three-dimensionalvirtual object from the first region into the second region of the spacethrough the display device, so that an image of the updatedthree-dimensional virtual object is generated in the second region ofthe space.
 3. The method according to claim 1, wherein the transmittingan optical signal of the three-dimensional virtual object to a firstregion of a three-dimensional space and projecting the optical signal ofthe three-dimensional virtual object from the first region into a secondregion of the space through the display device comprises: performing amirror flipping operation on the three-dimensional virtual object, toobtain a flipped three-dimensional virtual object; and generating anoptical signal of the flipped three-dimensional virtual object, andtransmitting the optical signal of the flipped three-dimensional virtualobject to the first region of the space, the display device including anoptical panel being disposed at a junction between the first region andthe second region of the space, so that the three-dimensional virtualobject is displayed in the second region of the space after the opticalsignal is negatively refracted by the optical panel.
 4. The methodaccording to claim 3, wherein the optical signal comprises a leftoptical signal and a right optical signal, and the generating an opticalsignal of the flipped three-dimensional virtual object, and transmittingthe optical signal of the flipped three-dimensional virtual object tothe first region of the space comprises: generating the left opticalsignal and the right optical signal of the flipped three-dimensionalvirtual object; and transmitting the left optical signal and the rightoptical signal to the first region of the space, so that the leftoptical signal enters a right eye of the user after being negativelyrefracted by the optical panel and the right optical signal enters aleft eye of the user after being negatively refracted by the opticalpanel.
 5. The method according to claim 2, wherein the transmitting anoptical signal of the updated three-dimensional virtual object to thefirst region of the space and projecting the optical signal of theupdated three-dimensional virtual object from the first region into thesecond region of the space through the display device comprises:performing a mirror flipping operation on the updated three-dimensionalvirtual object, to obtain a flipped updated three-dimensional virtualobject; and generating an optical signal of the flipped updatedthree-dimensional virtual object, and transmitting the optical signal ofthe flipped updated three-dimensional virtual object to the first regionof the space, the display device including an optical panel beingdisposed at a junction between the first region and the second region ofthe space, so that the updated three-dimensional virtual object isdisplayed in the second region of the space after the optical signal ofthe flipped updated three-dimensional virtual object is negativelyrefracted by the optical panel.
 6. The method according to claim 5,wherein the optical signal comprises a left optical signal and a rightoptical signal, and the generating an optical signal of the flippedupdated three-dimensional virtual object, and transmitting the opticalsignal of the flipped updated three-dimensional virtual object to thefirst region of the space comprises: generating the left optical signaland the right optical signal of the flipped updated three-dimensionalvirtual object; and transmitting the left optical signal and the rightoptical signal to the first region of the space, so that the leftoptical signal enters a right eye of the user after being negativelyrefracted by the optical panel and the right optical signal enters aleft eye of the user after being negatively refracted by the opticalpanel.
 7. The method according to claim 2, wherein the updating thethree-dimensional virtual object based on the motion sensing operation,to obtain an updated three-dimensional virtual object comprises:determining an image coordinate set of the three-dimensional virtualobject in a virtual coordinate system; generating, when the motionsensing operation of the user in the second region of the space isdetected, a motion sensing coordinate set of the user in the virtualcoordinate system based on the motion sensing operation; determining atouch status of the user for the imaging of the three-dimensionalvirtual object based on the motion sensing coordinate set of the userand the image coordinate set; and updating the three-dimensional virtualobject based on the touch status, to obtain the updatedthree-dimensional virtual object.
 8. The method according to claim 7,wherein the motion sensing coordinate set comprises a skeleton nodecoordinate set, and the generating, when the motion sensing operation ofthe user in the second region of the space is detected, a motion sensingcoordinate set of the user in the virtual coordinate system based on themotion sensing operation comprises: obtaining a hand image of the userin the second region of the space; performing skeletal analysis on thehand image, to determine skeleton nodes of a hand of the user; anddetermining coordinates of the skeleton nodes of the hand of the user inthe virtual coordinate system, to obtain the skeleton node coordinateset.
 9. The method according to claim 2, wherein the updating thethree-dimensional virtual object based on the motion sensing operation,to obtain an updated three-dimensional virtual object further comprises:determining motion sensing feedback information corresponding to themotion sensing operation; and transmitting the motion sensing feedbackinformation to the second region of the space, so that the user receivesthe motion sensing feedback information in the second region of thespace.
 10. The method according to claim 9, wherein the motion sensingfeedback comprises tactile feedback, the motion sensing feedbackinformation comprises an ultrasonic signal, and the determining motionsensing feedback information corresponding to the motion sensingoperation comprises: generating the ultrasonic signal based on themotion sensing operation; and the transmitting the motion sensingfeedback information to the second region of the space, so that the userreceives the motion sensing feedback information in the second region ofthe space comprises: transmitting the ultrasonic signal to the secondregion of the space, so that the user feels the tactile feedback formedby the ultrasonic signal in the second region of the space.
 11. Adisplay device, comprising: a display module, an optical panel, a motionsensing detection module, a memory, and a processor, the motion sensingdetection module being configured to detect a motion sensing operationof a user; the display module being configured to transmit an opticalsignal; the optical panel being disposed at a junction between a firstregion of a three-dimensional space and a second region of thethree-dimensional space and being configured to negatively refract anoptical signal of a three-dimensional virtual object transmitted to thefirst region of the space, so that an image of the three-dimensionalvirtual object is generated in the second region of the space by usingthe optical signal and the second region of the space being outside thedisplay device; the memory being configured to store data of thethree-dimensional virtual object and a plurality of instructions; andthe processor being configured to read the plurality of instructionsstored in the memory to perform the following operations: obtaining athree-dimensional virtual object; obtaining a three-dimensional virtualobject; and transmitting an optical signal of the three-dimensionalvirtual object to the first region of a three-dimensional space andprojecting the optical signal of the three-dimensional virtual objectfrom the first region into a second region of the space through theoptical panel, so that a three-dimensional image of thethree-dimensional virtual object is generated in the second region ofthe space that is outside the display device.
 12. The display deviceaccording to claim 11, wherein the processor is further configured toperform the following operations: when the motion sensing detectionmodule detects that the user performs a motion sensing operation on theimage of the three-dimensional virtual object in the second region ofthe space, updating the three-dimensional virtual object based on themotion sensing operation, to obtain an updated three-dimensional virtualobject; and transmitting an optical signal of the updatedthree-dimensional virtual object to the first region of the space andprojecting the optical signal of the updated three-dimensional virtualobject from the first region into the second region of the space throughthe display device, so that an image of the updated three-dimensionalvirtual object is generated in the second region of the space.
 13. Thedisplay device according to claim 11, wherein the transmitting anoptical signal of the three-dimensional virtual object to a first regionof a three-dimensional space and projecting the optical signal of thethree-dimensional virtual object from the first region into a secondregion of the space through the display device comprises: performing amirror flipping operation on the three-dimensional virtual object, toobtain a flipped three-dimensional virtual object; and generating anoptical signal of the flipped three-dimensional virtual object, andtransmitting the optical signal of the flipped three-dimensional virtualobject to the first region of the space, so that the three-dimensionalvirtual object is displayed in the second region of the space after theoptical signal is negatively refracted by the optical panel.
 14. Thedisplay device according to claim 13, wherein the optical signalcomprises a left optical signal and a right optical signal, and thegenerating an optical signal of the flipped three-dimensional virtualobject, and transmitting the optical signal of the flippedthree-dimensional virtual object to the first region of the spacecomprises: generating the left optical signal and the right opticalsignal of the flipped three-dimensional virtual object; and transmittingthe left optical signal and the right optical signal to the first regionof the space, so that the left optical signal enters a right eye of theuser after being negatively refracted by the optical panel and the rightoptical signal enters a left eye of the user after being negativelyrefracted by the optical panel.
 15. The display device according toclaim 13, wherein the updating the three-dimensional virtual objectbased on the motion sensing operation, to obtain an updatedthree-dimensional virtual object comprises: determining an imagecoordinate set of the three-dimensional virtual object in a virtualcoordinate system; when the motion sensing operation of the user in thesecond region of the space is detected by the motion sensing detectionmodule, generating a motion sensing coordinate set of the user in thevirtual coordinate system based on the motion sensing operation;determining a touch status of the user for the imaging of thethree-dimensional virtual object based on the motion sensing coordinateset of the user and the image coordinate set; and updating thethree-dimensional virtual object based on the touch status, to obtainthe updated three-dimensional virtual object.
 16. The display deviceaccording to claim 15, wherein the motion sensing coordinate setcomprises a skeleton node coordinate set, and the generating, when themotion sensing operation of the user in the second region of the spaceis detected, a motion sensing coordinate set of the user in the virtualcoordinate system based on the motion sensing operation comprises:obtaining a hand image of the user in the second region of the space;performing skeletal analysis on the hand image, to determine skeletonnodes of a hand of the user; and determining coordinates of the skeletonnodes of the hand of the user in the virtual coordinate system, toobtain the skeleton node coordinate set.
 17. The display deviceaccording to claim 12, wherein the updating the three-dimensionalvirtual object based on the motion sensing operation, to obtain anupdated three-dimensional virtual object further comprises: determiningmotion sensing feedback information corresponding to the motion sensingoperation; and transmitting the motion sensing feedback information tothe second region of the space, so that the user receives the motionsensing feedback information in the second region of the space.
 18. Thedisplay device according to claim 17, wherein the motion sensingfeedback comprises tactile feedback, the motion sensing feedbackinformation comprises an ultrasonic signal, and the determining motionsensing feedback information corresponding to the motion sensingoperation comprises: generating the ultrasonic signal based on themotion sensing operation; and the transmitting the motion sensingfeedback information to the second region of the space, so that the userreceives the motion sensing feedback information in the second region ofthe space comprises: transmitting the ultrasonic signal to the secondregion of the space, so that the user feels the tactile feedback formedby the ultrasonic signal in the second region of the space.
 19. Anon-transitory computer-readable storage medium, storing a plurality ofinstructions, the instructions, when executed by a processor of adisplay device, causing the display device to perform a methodincluding: obtaining a three-dimensional virtual object; andtransmitting an optical signal of the three-dimensional virtual objectto a first region of a three-dimensional space and projecting theoptical signal of the three-dimensional virtual object from the firstregion into a second region of the space through the display device, sothat a three-dimensional image of the three-dimensional virtual objectis generated in the second region of the space that is outside thedisplay device.
 20. The non-transitory computer-readable storage mediumaccording to claim 19, wherein the method further includes: when themotion sensing detection module detects that the user performs a motionsensing operation on the image of the three-dimensional virtual objectin the second region of the space, updating the three-dimensionalvirtual object based on the motion sensing operation, to obtain anupdated three-dimensional virtual object; and transmitting an opticalsignal of the updated three-dimensional virtual object to the firstregion of the space and projecting the optical signal of the updatedthree-dimensional virtual object from the first region into the secondregion of the space through the display device, so that an image of theupdated three-dimensional virtual object is generated in the secondregion of the space.